Hybrid Deep Learning Framework Research Articles

High rainfall event identification using remote sensing satellite data integrated with a hybrid deep learning framework

High rainfall event identification using remote sensing satellite data integrated with a hybrid deep learning framework

A hybrid deep learning framework for daily living human activity recognition with cluster-based video summarization

A hybrid deep learning framework for daily living human activity recognition with cluster-based video summarization

An integrated deep neural network model combining 1D CNN and LSTM for structural health monitoring utilizing multisensor time-series data

Introducing deep learning algorithms into the field of structural health monitoring (SHM) has contributed to the automatic extraction of damage-sensitive features, but the type and architecture of these algorithms are still in dispute. This paper proposes a hybrid deep learning framework entitled time-distributed one-dimensional convolutional neural network (1D CNN) long short-term memory (LSTM) model, which utilizes raw multisensor time histories to detect structural damages. Using a sliding window that moves along the temporal dimension, the multisensor data are first segmented into subsequences. The 1D CNN layers are simultaneously applied to each subsequence to extract damage-sensitive features from row data samples. These features are then fed into the LSTM layers to extract temporal features between subsequences. As the final step, these extracted features are classified using fully connected layers. In order to assess the performance of this model, a numerical model of a high-rise frame with nonlinear members is used. This hybrid model is assumed to identify the location of damages to this frame. In order to assess the proposed model with a real-world structure, a well-known benchmark building is employed to identify damage patterns by this deep hybrid neural network. A set of metrics related to the performance of the model is measured and evaluated. It is found that the model has an average accuracy of above 96.6% in localizing damage in the numerical structure and above 99.6% in detecting each damage pattern in the experimental building. The results indicate that the proposed model can be applied effectively to the SHM of different structural systems with different damage patterns.

BioDeepfuse: a hybrid deep learning approach with integrated feature extraction techniques for enhanced non-coding RNA classification

ABSTRACT The accurate classification of non-coding RNA (ncRNA) sequences is pivotal for advanced non-coding genome annotation and analysis, a fundamental aspect of genomics that facilitates understanding of ncRNA functions and regulatory mechanisms in various biological processes. While traditional machine learning approaches have been employed for distinguishing ncRNA, these often necessitate extensive feature engineering. Recently, deep learning algorithms have provided advancements in ncRNA classification. This study presents BioDeepFuse, a hybrid deep learning framework integrating convolutional neural networks (CNN) or bidirectional long short-term memory (BiLSTM) networks with handcrafted features for enhanced accuracy. This framework employs a combination of k-mer one-hot, k-mer dictionary, and feature extraction techniques for input representation. Extracted features, when embedded into the deep network, enable optimal utilization of spatial and sequential nuances of ncRNA sequences. Using benchmark datasets and real-world RNA samples from bacterial organisms, we evaluated the performance of BioDeepFuse. Results exhibited high accuracy in ncRNA classification, underscoring the robustness of our tool in addressing complex ncRNA sequence data challenges. The effective melding of CNN or BiLSTM with external features heralds promising directions for future research, particularly in refining ncRNA classifiers and deepening insights into ncRNAs in cellular processes and disease manifestations. In addition to its original application in the context of bacterial organisms, the methodologies and techniques integrated into our framework can potentially render BioDeepFuse effective in various and broader domains.

IC Packaging Material Identification via a Hybrid Deep Learning Framework with CNN-Transformer Bidirectional Interaction.

With the advancement of micro- and nanomanufacturing technologies, electronic components and chips are increasingly being miniaturized. To automatically identify their packaging materials for ensuring the reliability of ICs, a hybrid deep learning framework termed as CNN-transformer interaction (CTI) model is designed on IC packaging images in this paper, in which several cascaded CTI blocks are designed to bidirectionally capture local and global features from the IC packaging image. Each CTI block involves a CNN branch with two designed convolutional neural networks (CNNs) for CNN local features and a transformer branch with two transformers for transformer global features and transformer local-window features. A bidirectional interaction mechanism is designed to interactively transfer the features in channel and spatial dimensions between the CNNs and transformers. Experimental results indicate that the hybrid framework can recognize three types of IC packaging materials with a good performance of 96.16% F1-score and 97.92% accuracy, which is superior to some existing deep learning methods.

SatCoBiLSTM: Self-attention based hybrid deep learning framework for crisis event detection in social media

Social media platforms have emerged as vital sources of sharing real-time information during crises, enabling users to share critical details about disasters and ensuring timely awareness. This study addresses the daunting challenge of effectively detecting crisis events from a vast, noisy stream of short text data. To address the above-mentioned issue, we present a self-attention-based hybrid deep learning framework, SatCoBiLSTM, designed to extract hierarchical textual features and crucial information from textual data. The proposed model integrates a multi-scale convolution and BiLSTM layer to extract local and contextual relations, alongside a self-attention layer to select and focus on the most relevant crisis features. Various experiments have been conducted on three benchmark real-world crisis datasets to examine the proposed model’s performance. SatCoBiLSTM achieved an impressive F1-score of 96%, 94%, and 95% on the three public datasets. It also shows a promising improvement in the F1-score, compared to the state-of-the-art (SOTA) and baseline methods, by 1%, 1%, and 6%, respectively, showing its effectiveness in crisis event detection. An ablation analysis has been carried out to investigate the validity of each integrated layer in our model. The SatCoBiLSTM model’s capability to identify crisis-related information highlights its potential to enhance real-time awareness during disasters. Eventually, the study advances crisis event detection and lays the foundation for future research to process and handle short-textual data in noisy environments.

Decision Making in Monopoly Using a Hybrid Deep Reinforcement Learning Approach

Abstract: This research introduces a novel approach to decision-making in the classic board game Monopoly by leveraging a hybrid deep reinforcement learning framework. Traditional methods for decision-making in Monopoly often rely on heuristic strategies or rule-based systems, which may lack adaptability and fail to capture the dynamic and complex nature of the game. In contrast, this study proposes a hybrid model that combines the strengths of deep learning and reinforcement learning to enable an agent to autonomously learn effective decision-making strategies through interactions with the Monopoly environment. The hybrid deep reinforcement learning model is designed to process and interpret the intricate game state representations, learning optimal decision policies over time. The incorporation of deep neural networks allows the model to capture complex patterns and dependencies in the data, while reinforcement learning enables the agent to iteratively improve its decision-making abilities through trial and error. The research focuses on developing a suitable reward structure and feature representation that enhances the model's ability to navigate the diverse scenarios encountered in Monopoly, ultimately aiming to achieve superior decision-making performance compared to traditional approaches.

Enhancing Disease Prediction with a Hybrid CNN-LSTM Framework in EHRs

This study developed a novel hybrid deep learning framework aimed at enhancing the accuracy of disease prediction using temporal data from Electronic Health Records (EHRs). The framework integrates Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks, leveraging the strength of CNNs in extracting hierarchical feature representations from complex data and the capability of LSTMs in capturing long-term dependencies in temporal information. An empirical investigation on real-world EHR datasets revealed that, compared to Support Vector Machine (SVM) models, standalone CNNs, and LSTMs, this hybrid deep learning network demonstrated significantly higher prediction accuracy in disease prediction tasks. This research not only advances the performance of predictive models in the health data analytics domain but also underscores the importance of adopting and further developing advanced deep learning technologies to address the complexity of modern medical data. Our findings advocate for a shift towards integrating complex neural network architectures in developing predictive models, potentially offering avenues for more personalized and proactive disease management and care, thereby setting new standards for future health management practices.

Employing hybrid deep learning for near-real-time forecasts of sensor-based algal parameters in a Microcystis bloom-dominated lake

Harmful cyanobacterial blooms (CyanoHABs) are increasingly impacting the ecosystem of lakes, reservoirs and estuaries globally. The integration of real-time monitoring and deep learning technology has opened up new horizons for early warnings of CyanoHABs. However, unlike traditional methods such as pigment quantification or microscopy counting, the high-frequency data from in-situ fluorometric sensors display unpredictable fluctuations and variability, posing a challenge for predictive models to discern underlying trends within the time-series sequence. This study introduces a hybrid framework for near-real-time CyanoHABs predictions in a cyanobacterium Microcystis-dominated lake - Lake Dianchi, China. The proposed model was validated using hourly Chlorophyll-a (Chl a) concentrations and algal cell densities. Our results demonstrate that applying decomposition-based singular spectrum analysis (SSA) significantly enhances the prediction accuracy of subsequent CyanoHABs models, particularly in the case of temporal convolutional network (TCN). Comparative experiments revealed that the SSA-TCN model outperforms other SSA-based deep learning models for predicting Chl a (R2 = 0.45–0.93, RMSE = 2.29–5.89 μg/L) and algal cell density (R2 = 0.63–0.89, RMSE = 9489.39–16,015.37 cells/mL) at one to four steps ahead predictions. The forecast of bloom intensities achieved a remarkable accuracy of 98.56 % and an average precision rate of 94.04 % ± 0.05 %. In addition, scenarios involving various input combinations of environmental factors demonstrated that water temperature emerged as the most effective driver for CyanoHABs predictions, with a mean RMSE of 2.94 ± 0.12 μg/L, MAE of 1.55 ± 0.09 μg/L, and R2 of 0.83 ± 0.01. Overall, the newly developed approach underscores the potential of a well-designed hybrid deep-learning framework for accurately predicting sensor-based algal parameters. It offers novel perspectives for managing CyanoHABs through online monitoring and artificial intelligence in aquatic ecosystems.

A multi-channel hybrid deep learning framework for multi-sensor fusion enabled human activity recognition

Smart and connected health (SCH) accelerates the development and integration of information science and engineering approaches to support the digital transformation of health and medicine in populated societies. Sensor data based human activity recognition (HAR) as an effective means of SCH is promising for healthcare monitoring and ambient assisted living. This paper focuses on multi-position sensor data fusion enabled HAR, and proposes a multi-channel deep learning framework. The main contributions include: (1) A multi-channel hybrid deep learning model (1DCNN-Att-BiLSTM) that merges a one-dimensional convolutional neural network, a bidirectional long short-term memory model, and an attention mechanism is proposed to significantly improve the recognition performance of HAR models by extracting and selecting local and global behavioral features in the spatial and temporal domains; (2) Publicly accessible datasets Shoaib AR, Shoaib SA, and HAPT are collected and processed to build multi-position sensor data pool for model's evaluation; (3) Thoroughly evaluating the classification performance of seven sensor data fusion patterns from different body positions within the parallel network structure of the multi-channel framework; (4) Comparing the recognition performance metrics of traditional machine learning models, deep learning models, and hybrid models with our proposed model. Extensive experiments demonstrate that our approach achieves competitive performance.

Nondestructive detection of fiber content in steel fiber reinforced concrete through percussion method coordinated with a hybrid deep learning network

Accurate inspection of the steel fiber content is necessary to assure the designed performance and required quality of steel fiber reinforced concrete (SFRC). This paper presents a percussion-based nondestructive methodology in detecting the steel fiber content in SFRC components, incorporating a deep neural network capable of intelligent detection and automated classification of steel fiber content without the need for signal transformation or manual feature extraction. This research makes several noteworthy contributions. First of all, the percussion acoustic signals were captured on a smartphone from SFRC cubes with varying fiber volumes in a range of 0–2%. Subsequently, a hybrid deep learning framework named 1D CNN-BiLSTM, which harnesses advantages from both the one-dimensional conventional neural network (1D CNN) and the bidirectional long short-term memory (BiLSTM) network, was established and trained to classify these acoustic signals labeled with specific fiber contents. The results well manifest the excellent performances of the proposed network in classifying the steel fiber content, achieving a testing accuracy of 98%, which outperforms conventional machine learning methods, 1D CNN and 1D CNN-LSTM. Moreover, the noise resilience and adaptability of 1D CNN-BiLSTM were also greatly enhanced in contrast to these baseline methods. The testing accuracy exceeds 90.0% under strong noisy conditions in an SNR range of 2–10 dB. In addition, the trained model also presents an exceptional adaptability to identifying previously unseen signals from SFRC beams, yielding an accuracy of 90.34%.

A novel hybrid deep-learning framework for medium-term container throughput forecasting: an application to China’s Guangzhou, Qingdao and Shanghai hub ports

A novel hybrid deep-learning framework for medium-term container throughput forecasting: an application to China’s Guangzhou, Qingdao and Shanghai hub ports

Hybrid deep learning framework for weather forecast with rainfall prediction using weather bigdata analytics

Hybrid deep learning framework for weather forecast with rainfall prediction using weather bigdata analytics

Performance analysis of hybrid deep learning framework using a vision transformer and convolutional neural network for handwritten digit recognition

Digitization created a demand for highly efficient handwritten document recognition systems. A handwritten document consists of digits, text, symbols, diagrams, etc. Digits are an essential element of handwritten documents. Accurate recognition of handwritten digits is vital for effective communication and data analysis. Various researchers have attempted to address this issue with modern convolutional neural network (CNN) techniques. Even after training, CNN filter weights remain unchanged despite the high identification accuracy. As a result, the process cannot flexibly adapt to input changes. Hence computer vision researchers have recently become interested in Vision Transformers (ViTs) and Multilayer Perceptrons (MLPs). The shortcomings of CNNs gave rise to a hybrid model revolution that combines the best elements of the two fields. This paper analyzes how the hybrid convolutional ViT model affects the ability to recognize handwritten digits. Also, the real-time data contains noise, distortions, and varying writing styles. Hence, cleaned and uncleaned handwritten digit images are used for evaluation in this paper. The accuracy of the proposed method is compared with the state-of-the-art techniques, and the result shows that the proposed model achieves the highest recognition accuracy. Also, the probable solutions for recognizing other aspects of handwritten documents are discussed in this paper.•Analyzed the effect of convolutional vision transformer on cleaned and real-time handwritten digit images.•The model's performance improved with the implication of cross-validation and hyper-parameter tuning.•The results show that the proposed model is robust, feasible, and effective on cleaned and uncleaned handwritten digits.

CNN-based hybrid deep learning framework for human activity classification

CNN-based hybrid deep learning framework for human activity classification

EfficientShip: A Hybrid Deep Learning Framework for Ship Detection in the River

EfficientShip: A Hybrid Deep Learning Framework for Ship Detection in the River

CNN-based hybrid deep learning framework for human activity classification

CNN-based hybrid deep learning framework for human activity classification

A hybrid deep learning framework for conflict prediction of diverse merge scenarios at roundabouts

The unique traffic situation at roundabouts causes complex interactions between merging vehicles, thereby increasing the likelihood of conflicts. Reliable prediction of conflict risk contributes to active safety improvement, but few studies have investigated the merge risk of roundabouts at a microscopic level. In light of this, this study develops a hybrid deep learning framework for predicting potential conflict risks in complex merging scenarios at roundabouts. Specifically, a roundabout coordinate system is devised to define vehicle characteristics based on trajectory data. Then, an improved 2D-TTC (time-to-collision) indicator is employed to identify two-dimensional merge conflicts. Since the surrounding vehicles may change as vehicles merge into a roundabout, this study analyzes several merging scenarios involving different vehicle groups and conflict durations in order to provide a comprehensive understanding of the conflict mechanism. For these scenarios, a hybrid model consisting of a convolutional neural network (CNN) and a long short-term memory network (LSTM) integrated with the convolutional block attention module (CBAM) is utilized to identify key features. The superiority of the proposed prediction method is demonstrated in comparisons with benchmark models. Results showed that segmental predictions were more accurate than overall predictions in terms of conflict duration. Furthermore, it is possible that a specific vehicle group has a decisive effect on the merging conflict risk, as indicated by the fact that information from multiple vehicle groups does not significantly improve the prediction performance. Another finding is that the driving state of vehicles merging at the roundabout varies considerably, but rarely with consecutive or multiple changes. The study provides novel insights into roundabout conflict prediction, which could serve as a tool for enhancing safety management involving complex traffic scenarios.

Exploring quantitative assessment of cybersickness in virtual reality using EEG signals and a CNN-ECA-LSTM network

Cybersickness in the virtual reality (VR) environment presents a significant challenge for user experience. Accurate evaluation methods are essential to mitigate or prevent cybersickness for the users. Compared to traditional subjective user studies such as questionnaires, electroencephalography (EEG) has the advantage of high temporal resolution, enabling more precise real-time evaluation. In this paper, we propose a novel hybrid deep learning framework that integrates a convolutional neural network (CNN), efficient channel attention (ECA) module, and long short-term memory (LSTM) to evaluate VR cybersickness using EEG signals quantitatively. A novel experimental paradigm is developed to induce cybersickness through virtual head rotation. A total of thirty-six subjects were recruited for the experiment, and both EEG signals and subjective questionnaire data were collected accordingly to build a cybersickness-EEG dataset. The proposed framework enables a meaningful evaluation of cybersickness, and the experiment demonstrates the framework’s effective ability to decode the correlation between EEG and cybersickness. The relevance between EEG signals and model predictions is also investigated to demonstrate the interpretability and effectiveness of the proposed framework. Our findings reveal a robust correlation between cybersickness and heightened activation intensity in the occipital and temporal areas.

Recognizing sports activities from video frames using deformable convolution and adaptive multiscale features

Automated techniques for evaluating sports activities inside dynamic frames are highly dependent on advanced sports analysis by smart machines. The monitoring of individuals and the discerning of athletic pursuits has several potential applications. Monitoring individuals, detecting unusual behavior, identifying medical issues, and tracking patients within healthcare facilities are examples of these applications. An assessment of the feasibility of integrating smart real-time monitoring systems across a variety of athletic environments is provided in this study. Motion and activity detection for recording sporting events has advanced due to the need for a large amount of both real-time and offline data. Through the use of deformable learning approaches, we extend conventional deep learning models to accurately detect and analyze human behavior in sports. Due to its robustness, efficiency, and statistical analysis, the system is a highly suitable option for advanced sports recording detection frameworks. It is essential for sports identification and administration to have a comprehensive understanding of action recognition. An accurate classification of human activities and athletic events can be achieved through the use of a hybrid deep learning framework presented in this study. Using innovative methodologies, we conduct cutting-edge research on action recognition that prioritizes users’ preferences and needs. It is possible to reduce the error rate to less than 3% by using the recommended structure and the three datasets mentioned above. It is 97.84% accurate for UCF-Sport, 97.75% accurate for UCF50, and 98.91% accurate for YouTube. The recommended optimized networks have been tested extensively compared to other models for recognizing athletic actions.